Universal and Succinct Source Coding of Deep Neural Networks

Deep neural networks have shown incredible performance for inference tasks in a variety of domains, but require significant storage space, which limits scaling and use on-device intelligence. This paper is concerned with finding universal lossless compressed representations deep feedforward synaptic weights drawn from discrete sets, directly performing without full decompression. The basic insight that allows less rate than naïve approaches recognizing the bipartite graph layers kind permutation invariance to labeling nodes, terms inferential operation. We provide efficient algorithms dissipate this irrelevant uncertainty then arithmetic coding nearly achieve entropy bound manner. also experimental results our approach on several standard datasets.